Utility maximising stochastic dynamic programming: An overview

Many real world problems involve making repeated decisions over time in an uncertain environment. These decisions often involve a trade-off between some immediate benefit(s) and possible future benefit(s), and also take in to account the impact the decision will have on future decisions and benefits. Stochastic dynamic programming (SDP) is often used to analyse problems of this type and the objective is often to maximise the expected value of benefits, which can imply that the decision-maker is ‘risk neutral’. But is this appropriate? In this paper a SDP formulation is described which accommodates risk attitudes via a utility function. The approach is discussed and illustrated for stochastic reservoir management and stochastic route choice problems.